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THE USEFULNESS OF CASE STUDY RESEARCH 
IN THE STUDY OF A MOTIVATIONAL INTERVENTION 



by 



Jan Visser 



ABSTRACT 



This paper discusses the methodological aspects of a study, the content and results 
of which have been presented more comprehensively^ and with different objectives, elsewhere 
(Visser, 1990). Conclusions focus therefore particularly on design features and their 
effectiveness of the case study approach applied in conducting the study, as well as on questions 
of scientific rigor in that connection. 

The paper reportr< on a disciplined inquiry into the area of motivation to learn and 
systematic ways of influencing learners in this regard. Learners, in adverse motivational 
circumstances, were prompted, by means of, usually wri:ten, messages, normally delivered 
outside the classroom periods, to adjust their disposition to undertake learning tasks. Such 
messages were designed and used according to a systematic, replicabie process. 

The study identifies a naturalistic moje of inquiiy as the most appropriate one for 
the particular research interest. Within that mode, it calls for a case study approach, from an 
embedded multiple-case perspective. 

Three basic research interests were addressed in the study, i.e. that of the validity of 
the motivational .messages strategy under consideration, the question as to how and whj the 
messages work, and the issue of their optimality. An extensive data base was established which 
could be analyzed with a view to substantiating or insubstantiating research propositions related 
to the three basic research interests. The proposition that the motivational messages strategy 
worked similarly well in the case of two of its applications almost three years apart was found 
substantiated by the available evidence. Of a total of ten prepositions regarding the interaction 
between the messages and the learners, the messages and the instructor, and the messages and 
the environment, nine were found substantiated and one insubstantiated in the light of the data 
collected. Evidence also supported the contention that the strategy adapts itself to the 
circumstances of its application. 

As to the overall interpretation of the totality of phenomena discussed, the study 
concludes that motivational messages should not be looked upon in isolation, but as an 
integrating part of a more complex motivational system^ in the context of which they enhance 
the effectiveness of the other components involved. 



llie research reported on in this paper was carried out in Mozambique. While it v/as conducted for its own sake, to serve as the basis 
for improvement of instructional practice, it was at the same time taken advantage of for a di^rtation thesis, defended at the I^orida 
State University, Department of Educational Research, Instructional Systems Program, Tallahassee, Florida, 32306. 
Tlic author's present address is: UNESCO Office Harare, P» O. Box 4775, Harare, Zimbabwe. Telefax: (263-4) 729253. 
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INTRODUCnON 



PURPOSE 

In the present paper I shall analyze the usefulness of applying a case study approach in 
researching the effectiveness of a motivational intervention in an instructor-facilitated learning context. 
The motivational intervention in question was one in which learners were prompted into modifying their 
disposition to engage in learning tasks by administering to them, regularly and frequently, so-called 
motivational messages (Visser, 1988). The usual format of motivational messages used is that of a letter 
size sheet of paper folded twice in directions perpendicular to each other. Folding the paper once 
produces a double piece of paper, the size of which is half that of the original, that can be opened and 
closed along the second fold. It therefore has a cover and an inside, each of which can be used in 
different ways. The cover would normally contain some kind of logo representative of the learning task 
in question, e.g. the name of the course, and a graphic or verbal attention getter. The inside was used 
to transmit the main motivational content, which could be of a general nature, addressing all 
participating learners in exactly the same way, or it could be personalized, or made personal. The 
content of each message was designed so as to respond adequately to the motivational state learners 
were found to be in, as determined on the basis of continuous analysis of the motivational behavior of 
the learners in terms of the ARCS model (Keller, 1987a&b), i.e. interpreting observations regarding the 
students along the dimensions of Attention, Relevance, Confidence, and Satisfaction. The purpose of 
this paper is in the first place to present the various features of how the study was carried out and to 
discuss these features in terms of the requirement of scientific rigor. A brief account will also be given 
of the results of the study, particularly wth a view to elucidating the breadth of information that can be 
obtained in one single study. A more complete account of the results can be found elsewhere (Visser, 
1990). 



PROBLEM 



In the context of the purpose of the present paper, the problem addressed by the study reported 
on was a two-fold one. In the first place there was the actual reseaich problem, reflected in the 
research questions to be answered by the study. The second problem was a methodological one. Could 
the method used to address the research questions be shown to be effective? It is particularly this last 
question that will receive attention here. However, in order to discuss the effectiveness of how 
something was done, one has to look as well at what was done and for what reason it was done. It is 
simply a question of means and ends. Particular modes of inquiry and specific research methods are 
used and can be effective for certain well defmed problem areas, they may fail to work in other areas. 

When the study under discussion in this paper started, the following situation prevailed: 

1) The strategy called the clinical use of motivational messages had once been used in the framework 
of an instructor-led course that had simultaneously .served as a basis for its development and the 
preliminary testing of its effectiveness in helping learners in adverse motivational circumstances to 
cope with their motivational restraints (Visser & Keller, 1989). While a study of the validity of a 
strategy in circumstances in which that strategy was developed as a means to cope with these 
circumstances is certainly useful, it cannot be denied that one capitalizes heavily on chance if one 
v^shes to generalize from such circumstances and claim success beyond that single event. 

2) A new occasion had presented itself, several years separated from previous one, in which a course 
would have to be taught to students in circumstances that should be expected to hamper 
significantly their disposition to undertake and persevere in a difficult learning task. The course 
in question was one in instructional design technology. It was taught to personnel of different 
entities pertaining to one particular ministry in Mozambique. The adverseness of the motivational 
conditions in these circumstances will be analyzed later. The clinical use of motivational messages 
was now a candidate strategy to solve the motivational problem existing in connection with that 
second course. It was selected, and now the question was, will it work again? The opportunity to 
carry out a second study concerning the same strategy was seized upon as a possibility to either 
disconfirm the previous findings or to strengthen their validity, depending on what the outcome of 
the second study would be in comparison with the first one. Either outcome would be of interest 
from a research point of view. 

3) The person asking the above question, and therefore intending to carry out a study about it, 
happened to be the same one who had developed the motivational strategy in question and who 
was requested to teach the course in which the strategy was selected to be applied. This introduced 
a considerable risk of researcher bias, that should be eliminated in designing the study. A 
particular question of interest in that connection was whether the clinical use of motivational 
messages could be looked upon independently from the instructor who used it. 

The kick-off point for the study being as described above, the central problem facing the 
researcher was: Can a sound form of inquiry be designed through which an.swers can be obtained 
regarding the effectiveness of the motivational messages strategy and of the underlying motivational 
problem solving analysis tool, Keller's (1984, 1987a&b) ARCS model, as well as regarding the conditions 
under which the strategy works and why and how it works? 



METHOD 



MODE OF INQUIRY 

In selecting the mode of inquiry for the study envisaged, it was considered that: 

* the application of the clinical use of motivational messages is a complex matter, involving a v/ide 
range of interaction patterns between instructor and students, in a context in which learner 
motivation can be affected by an almost infinite number of external factors 

* the clinical use of motivational messages Involves a process which assumes that the instructional 
facilitator concurrently adopt the role of motivational facilitator, i.e. by interacting with the learners 
in a variety of ways s/he monitors changes in the motivational variables of the learners, while, 
through the same channels, s/he provides feedback to the learners that helps them overcome their 
motivational restraints. 
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Considering the above, the focal point of the research interest was the process, rather than just 
the final result. In other words, if anything was to be generalized from the study that generalization 
should be based on understanding the mechanics underlying the process involved in applying the 
motivational messages strategy. That process is characterized by the development over time, within 
changing contexts, of complex patterns of interaction between inputs, actions, outputs and feedback. 
Stake (1978, 1986, 1988) refers in this context to naturalistic generalization. 

The crucial role o( pattern and context in achieving knowledge" has been identified by Campbell 
(1984, p. 9) as an important reason to adopt research modes that go beyond the traditional laboratory 
settings, particularly those that involve conducting case studies. Others, e.g. Guba (1981) and Cuba & 
Lincoln (1981, 1982), have emphasized that the concept of disciplined inquiry (Cronbach & Suppes, 
1969) does not necessarily refer to the exclusive use of the rationalistic paradigm - as much of the 
existing educational research base would perhaps want one to b<;lieve - but that it can just as well be 
based on a naturalistic paradigtn. Various reviewers of existing research in different areas, e.g. Clark 
& Angert (1979) and Schramm (1977) have pointed to the wastefulness of research that does not attend 
to "the totality of the learning environment" (Clark and Angert, p. 7) or, in Schramm's critique of media 
research, that does not aim at "trying to identify the unique strengths and weaknesses of a given medium 
for a given purpose, and [that is not conducted with a view to] trjdng to maximize the learning from a 
particular mediurn and thus considering how it can be used and how it can be used best (p.36, the 
former emphasis is mine, the latter two are Schramm's). The idea of maximizing learning in a given 
purposeful context as an important research concern is taken up again, more than ten years later, by 
Reigeluth (1989). He urges researchers to "place greater emphasis on optimality as opposed to validity" 
(p. 73, my emphasis). Again, optimality is a process related concc.pt. It has to do with the possibility 
to show that the way an intervention is used has built into it adequate mechanisms for its continued 
improvement, i.e. for its iterative effectiveness. 

All the above concerns point to the appropriateness of a naturalistic mode of inquiry for the 
research in question. Visser (1990) discusses that the philosophical underpinnings of this emerging 
research paradigm in the study of instructional processes have long been recognized as the basis for 
important developments of the natural sciences, particularly in twentieth century physics, concluding that 
these developments should have increased our awareness "the importance that should be attached to the 
reality of description, rather than to the description of reality" (p. 34). The use of the naturalistic paradigm 
leads to a fullness of description, often in complementary terms, which it would be impossible to attain 
under the rationalistic paradigm. 



TYPE OF RESEARCH STRATEGY 

Yin (1984) lists the following three criteria for the choice of an appropriate research strategy: 

a) ihe type of research question asked 

b) the degree of control that can be exercised over behavioral events in the research context 

c) the question whether the research deals with contemporary or with historical events. 

The research envisaged, as indicated in the introduction, concentrated on contemporary events 
in a real-life context, i.e. in a setting in which virtually no control existed over behavioral events. 
Besides, the research questions asked in that context attached particular importance to the how and why 
of what happens as a result of the clinical use of motivational messages. Yin (1984) points out that 
under such circumstances a case study approach is typically called for. However, before proceeding to 
analyze what specific kind of case study design would be the most adequate one for the research 
concerned, the research questions, as well as the ensuing research propositions, should be discussed in 
more detail. 



SPECMC RESEARCH QUESTIONS AND RESEARCH PROPOSITIONS 

It has been mentioned betorc that the motivational intervenrion which was the object of the 
study had been researched on an earlier occasion, i.e. in the instructional context in which it had been 
developed. This being .o, an excellent opportunity existed to use the data base pertaining to the earlier 
study as a frame of reference for comparison with data to be collected in the context of the study under 
discussion. A first research question was therefore related to that particular interest. Two other 
research questions were added to the first one, particularly to explore the how and why aspects of the 
clinical use of motivational messages, and to address the question of optimality as mentioned before. 
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The three questions and their related research propositions , the latter sometimes summarized, are 
presented below. 

Tabic 1 

Research questions and their related research propositions* 



RESEARCH QUESTION 



RESEARCH PROPOSITION 



1. In applying the motivational strategy 
under scrutiny in the given instructional 
context^ will similar patterns of data 
regarding the relation between the clinical 
use of motivational messages and the 
capability of learners to cope with their 
motivational restraints be found as those 
that emei^ed from the previous study. 

2. How and why does the clinical use 
of motivational messages work? 



3. Can it be shown that the clinical use 
of motivational messages is iterativcly 
effective? 



1.1.1 The clinical use of motivational messages in the given 

instructional context will be similarly effective as in the case of the 
previous study. 



Motivational messages work because they have the following effects 
on the learners; 

2.1.1 They help learners assess their own progress. 

2.1.2 They remind them of goals and intermediate goals. 

2.1.3 They allow them to correct their own study behavior. 

2. 1 .4 They enhance their perception of self-control. 
2.1J5 They prompt them to optimize their time-on-tash 

2. 1 .6 They enhance their satisfaction about accomplishments. 

Motivational messages work because they have the following effects 
on the instructor^ influencing his/her face-tO' face interaction with the 
students: 

2.2.1 Vteir design and delivery process enhances the teacher's 
awareness of students' actual and potential problems. 

2.2.2 They prompt the instntctor into adopting a continuous 
mode of being motivational, thus influencing his/her in- 
class behavior. 

Motivational messages work because they have the p illowing effects 
on the instructional environment: 

2.3.1 They enhance the instructional environment directly, as 
they contribute to students and instructor feeling 
themselves jointty responsible for that environment. 

2.3.2 They enhance the instructional environment indirectly 
through changes brought about in that environment as 
a result of the messages' content. 

3. 1 . 1 The clinical use of motivational messages is continually 
effective over time. (This means that the strategy, by virtue of the 
way its application is structured, adapts itself to changing 
circumstances, always seeking to attain optimal effectiveness.) 



UNIT OF ANALYSIS 

The research questions, as formulated above, were based on the general proposition that a 
relation exists between, on the one hand, the clinical use of motivational messages in a given instructor- 
facilitated learning context, characterized by adverse motivational circumstances, and, on the other hand, 
the capability of learners to overcome their existing motivational restraints. The interaction between the 
motivational messages and the learners* capability to cope with the adverse motivational conditions of 
the instructional environment was analyzed in the study at the levels of 

- perceived motivational status 

- perceived motivational needs 

- learning behavior (process) 

- instructional attainment and achievement (product), 
attending to both group and individual aspects. 

Analysis was structured such that generalizations from the case - naturalistically (Stake, 1988) 
to other applications and analytically (Yin, 1984) to theory, but certainly not statistically to populations - 
could be made fundamentally in terms of similar, but not necessarily identical technology that could be 
developed and used according to the same principles. 
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TYPE OF RESEARCH DESIGN 

Yin (1984) classifies types of case study design along two dimensions, i.e. single-case vs. multiple- 
case and holistic vs. embedded. The research activity under discussion in this paper fitted in that scheme 
as follows. In terms of data collection it was an embedded single-case, considering that data of the 
previous case were ? ^rcady there, and that both the data of the past study and those collected for the 
present one reflectec on diverse subunits of analysis. On the other hand, in terms of data analysis, 
particularly as regards the first research question, the data collected in the framework of the study under 
consideration were looked upon within an embedded multiple-case perspective, which encompassed both 
the previous study (Visser & Keller, 1989) and the case under consideration. 

RATIONALE FOR SELECTION OF THE CASE 

Considering that literal replicaticn CYin, 1984) was one of the aims of pairing the previous case 
with the one under discussion, the rational for selecting the second case had been the same as that for 
the previous one. Like on the previous occasion, the new case had been chosen on the basis of the 
consideration that it was a critical one (Yin, 1984). The motivational conditions present in the 
instructional environment were so adverse that it would be difficult to explain motivational effects in ways 
different from those that could be related to the motivational intervention which was the object of the 
study. 



RESEARCH SETITNG AND SUBJECTS 

As mentioned earlier, the setting for the study reported on in this paper was an instructional 
design course for the training of personnel pertaining to different entities of a particular ministry in 
Mozambique. The course expected of the learners that they would engage, during a period of at least 
three months, in an intensive program of autonomous study, both individually and as a group, involving 
work in compulsory classroom sessions as well as,, to a large extent, outside the classroom. Students 
were furthermore expected to complete an individual instructional design project, the product of which 
should be a developed and formatively tested instructional module, as well as an extensive report 
documenting the entire design process. Strict performance referenced evaluation criteria were set for 
the above. 

Particularly adverse motivational conditions surrounded the course, such as absence in the 
learners of a positive individual choice to participate in the course; lack of individual commitment; 
absence of material rewards; distracting difficult material circumstances; a dwindling socio-ideological 
frame of reference, related to profound changes taking place in society at the time the course was given, 
causing confusion and lack of stable subsumption of individual motives in societal aspirations; dissonance 
with expectations based on past, normally far less demanding, experience of participating in courses; 
discrepancy between course demands and normal evnectations as regards performance in the job context; 
incompatibility of course demands with professional and personal obligations; uneasiness felt by the 
learners because of the expectations inherent in the course objectives that their work routine should 
change in the future; frustration related to the frequently to be expected consequence of formative 
testing of the instructional modules under development to start all over again; possible threat to their 
authority for those participants who were taking part in the course together with personnel whose work 
they were supposed to guide; unfamiliarity of the Portuguese speaking audience with the Spanish 
language in which a large part of the instructional materials was written; and excessive demands on 
individual and collective inventiveness to solve organizational aspects of the design projects. This list, 
although extensive, should not be interpreted as exhaustive. 

Subjects participating in the course constituted a group the composition of which had come 
about naturally in the context in which the need for the course had been identified. Employees of the 
different entities of the ministry in question had been selected on the basis of the relevance of the 
expected improvement of their instructional design capabilities for their professional performance. The 
choice had normally been the employer's. His or her perception of what was good for the employee 
should not be expected to coincide with a need felt by the employee. The researcher himself had also 
been appointed to U ich the course independently from his research interest. This put him in an ideal 
position to take on, together with two of his students, participant observer roles. 
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The group of participants in the course, who were therefore subjected to the motivational 
intervention under consideration, included 22 regular students, for whom no particular limitations were 
known at the time of their admission as to their possibilities to complete the course, 7 special students 
(including two research assistants, performing participant observer roles), for whom it was expected that 
their regular work conditions would not allow them to fulfill all course requirements, 2 invitees, who 
were allowed to participate in activities of their choosing, without any obligation to fulfill particular 
course requirements, and 1 observer, who entered the course after a considerable part of it had already 
been completed, and who was allowed and expected to attend all remaining course activities. His status 
was later changed in that of a special student. 

The 29 regular and special students considered for the study had a bimodal age distribution, with 
most participants in the age group between 25 and 30 years (M = 31.2 years, S.D. = 7.2 years). Their 
gender distribution was 19 male (66 %) and 10 female (34 %). The number of years during which they, 
had received professional schooling, including university level training, showed a bimodal distribution, 
with ahnost equal groups having up to three years or between four and seven years of training (M = 
4.1 years, S.D. = 2.0 years). Seven of them held university degrees at the bachelor level, and seven at 
the level of a licenciatura, a degree obtained after 5 years of university training. Their content 
specializations varied widely. Four of them had Portuguese, the official language of Mozambique, as 
a mother tongue, whereas two grew up bilingually, with Portuguese as one of the local languages spoken 
at home. Twenty had a local language as mother tongue, and three were speakers of languages foreign 
to the Mozambican environment. As to their demographic background, 10 were rural, 6 came from 
small provincial towns, and 13 had an urban background. Of the total of 29, 26 were of Mozambican 
nationality. For most of them their course participation came a considerable number of years after the 
lasr occasion on which they had received training where performance standards had to be reached. For 
three of them this period was more than ten years, for another three it was between five and ten years, 
for 14 between two and five years, and for nine less than two years. 

The group described above had characteristics largely similar to the one used for the previous 
study (Visser & Keller, 1989), except that for the new stu^ the group was twice as big as the previous 
one, a larger proportion held university degrees, less students had Portuguese as mother tongue, their 
demographic background was to a somewhat larger extent urban, and the period of time lapsed since 
their last experience with in course in which performance standards had to be met was slightly less. 

It should also be observed that the group of subjects used for the second study was .made up 
of different subgroups, pertaining to the different entities of the ministry in question. There was 
therefore far less cohesion between the students at the start of the second course than had been the case 
in the previous study. One of the subgroups, on the other hand, the largest one, showed particular 
similarity with the totality of the subjects used in the previous study. 

The existence of both similarity and divergence of characteristics between the groups of subjects 
used in the two different cases created excellent conditions for the type of comparison referred to in the 
first research question, providing an opportunity for strengthening the validity of the motivational 
intervention as well as for broadening the basis of its naturalistic generalization. 



DATA COLLECTION 

In order to enhance the possibility of corroboration of data obtained from different sources and 
by different means, a complex system of data collection methods was used. Fundamental in that process 
was the role played by three participant observers. One participant observer was the lead researcher, 
whose research role was embedded within his function as instructor and motivational facilitator of the 
course. The other two participant observers were both students of the course. While students of the 
course were made aware, in the context of the course content, that research of any instructional acti nty, 
including the course itself, was a natural part of the instructional designer's concern, all specific tasks 
of participant observation were performed in a totally unobtrusive way. In a debriefing after the course, 
the role of the two student participant observers was finally revealed to the other participating students. 
The occasion constituted an opportunity to establish that the objective of complete unobtrusiveness had 
indeed been attained. 

The two student participant observers went through a one-month training/briefing period 
preceding the -ourse, in which they enrolled as real students. Apart from their observations regarding 
their colleagues, they could also observe their own behavior and that of their fellow participant observer. 
Besides, independent observation of and reporting on (he instructor's behavior had been included as a 
fundamental part of their tasks. 
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Their incorporation in the research effort played a vital role in enhancing the internal validity 
and reliability of the study. With that aim in mind the process of their participation was structured such 
that, after their training, they were working on the basis of research protocols established during the 
training process, in a way independent of the lead researcher. Their observations have resulted in an 
independent research assistants' report which is included in the data base of the study. 

The joint effort of the research team (lead researcher plus two research assistants) concentrated 
on the following data sources: 

1) Open-ended anonymous questionnaires, administered to participants on a regular, usually weekly 
basis. Information obiained through these questionnaires also played a role in the design process 
of the motivational messages. 

2) Three end-of-course questionnaires that were either anonymous or, if not anonymous, unobtrusive, 
i.e. hidden in a course exercise. On two of these questionnaires students rated the importance they 
attributed to the motivational messages (designated as **little notes", the name used by the students 
themselves) among other components of the instructional strategy of the course as well as in 
relation to other media used. Besides numerical rating of perceived importance, the audience was 
requested to justify their answers. The third questionnaire asked participants to list the five most 
important things they had learned during the course. 

3) An open-ended discussion about the course during the last class session. 

4a) Lead researcher registration of spontaneous and solicited remarks by students and other individuals 

interacting wth the course environment. 
4b) Independent research assistant registration of the type of remarks referred to under 4a. 
5a) Recording by the lead researcher of immediate, observable effects of particular messages. 
5b) Independent recording by the research assistants of the type of effects referred to under 5a. 
6a) Lead researcher observation of participants' reactions and attitudes towards the medium as such. 
6b) Independent research assistant observation of reactions and attitudes referred to under 6a. 

7) Instructional outcome of the course, both in terms of attainment and achievement. 

8) Pre-instructional and structured j^ost-instructional interviews with course participants. 

9) Research assistant observation, as well as systematic audio Recording, of teacher behavior. 

Inclusion of the research assistants* findings in the data base followed a process of initial 
briefing, i.e. the one-month training period mentioned before, and subsequent research debriefing 
sessions held at regular, usually weekly, intervals. Research debriefings covered the whole period of the 
course, as well as a considerable period after it, to attend to follow-up interests and to monitor post- 
instructional impact. A period of six months elapsed between initial briefing and final debriefing. 

Research debriefings generally followed the model of initial reporting by both research assistants 
of their individual findings, interpretative discussion of these findings on the basis of the pre-established 
research propositions, discussion of possible alternative interpretations, possible re-interpretation of past 
conclusions in the light of the expanding data base, and planning of follow-up activities. In the interest 
of validity and reliability, all research debriefings were recorded on magnetic tape. They have as such 
been included in the database. Besides, type written accounts based on these recordings have also been 
included in the data base. For the sake of completeness, it is recalled here that the research assistants 
produced, at their own initiative, an independent report, likewise included in the data base, describing 
their involvement in the research effort, analyzing their findings, and discussing a number of their 
conclusions about the effectiveness of the motivational messages strategy. Particular attention was paid 
in this report to the need for independent reporting on the role of the instructor, as opposed to the lead 
researchers own informed perception of his role as such. 



DATA ANALYSIS PROCEDURE 

Prior to the data collection it had been established how data obtained from the various sources 
would respond to the various research questions, by either substantiating or insabstantiating the different 
research propositions. The overview presented on the next page shows the connection. 

The way the data sources were intended to be used in relation to the different research 
questions depended on these questions and their specific research purposes - validation, explanation, or 
exploration. 

In connection mth Research Question 1, which concentrated on comparison with the previous 
case (Visser & Keller, 1989), it is observed that analysis of the new data had to follow a process similar 
to the one used previously, so that the description of the two cases would be in commensurate terms, 
and the comparison could hence be made on the basis of pattern-matching (Yin, 1984). 
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Tablc2 

Overview of the relation between research questions and particular data sources planned to be used for 
the purpose of substantiating or insubstantiating related research propositions. 



Data Sources 1, 2, 3, 4a, 5a, 6a, 7 to be used to analyze the 
extent to which the clinical use of motivational messages is 
similarly effective in the proposed case as in the previous 
case. 

Data Sources 4b, 5b, 6b, 8, 9 to be used additionally, 
particularly for corroboratory purposes. 

Data Sources 1, 2a+b [open-ended part]. 2c, 3, 4a, 4b, 5a, 

5b, 8 to be used for e:q)lanation building. 

Data Source 9 to be used additionally for corroboration. 

Data Sources 1, 2a+b [both structured and open-ended 
parts], 2c, 3, 4a, 4b, 8, and particularly 9 to be used for 
explanation building. 

Data Sources 1, 2a+b [open-ended part], 2c^ 3, 4a, 4b, 5a, 
5b, 6o^ 6b, 8) 9 to be used for explanation building. 



Data Sources 1, 2a+b [particularly open ended part], 3, 4a, 
4b, 5a, 5b, 6a, 6b, 7, 8, 9 to be used for assessment of 
continuous effectiveness of motivational strategy over time 
and adaptability to changing circumstances. 



Specific techniques in connection with data analysis related to the first research question 
concentrated on: 

content analysis (e.g. Fetterman, 1989) of the weekly questionnaires, the three end of course 
questionnaires, the open-ended discussion at the end of the course, md the post-instructional 
structured interviews (Data Sources 1, ?a,b,c 3, and 8) 

statistical processing of numerical data obtained through responses to the end-of-course 
questionnaires (Data Sources 2a„b) 

analysis of tendencies and patterns over time and across learners of students' remarks, observed 
effects of the messages, and students' reactions and demonstrated attitudes towards the medium 
(Data Sources 4a,b, 5a,b, and 6a,b) 

analysis of statistical indicators of student achievement and of compliance by students with deadlines 
(Data Source 7) 

analysis of convergence/divergence between observational findings reported from different angles 
(Data Source 9). 

Using the above techniques, and combining the results of the analysis of the different aspects 
concerned, a synthetic description could be obtained, providing the broad picture within which the 
various details referred to above could be integrated. That broad picture could then be compared with 
the description resulting from a similar analysis of the previous case (Visser & Keller, 1989). That 
comparison, in turn, pursued situating the two cases in a yet more general framework that could account 
for convergence as well as inconsistency and contradiction both between and within the tv/o cases 
(Mathison, 1988). 

The second research question was of particular importance from the point of view of explanation 
building (Yin, 1984), thus calling for content analysis of spontaneous and solicited information provided 
by students and of observations recorded by the research team. The different sets of propositions related 
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to this research question represent potentially differenf explanations for the way in which the clinical use 
of motivational messages may or may not affect the learner's motivational behavior. Convergence, 
consistency, and contradiction of the collected data as regards these sets of propositions were explored 
as a means to assess rivalry and complementarity of propositions. Yin (1984) emphasizes in this 
connection the "iterative nature of explanation-building" (p. 108). In bmKling towards the explanation 
one goes through a series of successive steps in which the comparison of findings wth initial propositions 
may lead to revising the propositions and consequently comparing further details of the case against the 
revised propositions, repeating this process as many times as necessary. The process of continuous 
research debriefings and preparation of subsequent research tasks was a powerful means to ensure this 
iterative cliaracter of the knowledge-building process. 

The third research question called for time-series analysis (Yin, 1984) of relevant information 
provided by the various data sources addressing that question. The exploration of both quantitative data, 
e.g. those regarding timely completion of learning tasks and achievement obtained (Data Source 7) and 
qualitative data from interviews, questionnaires, and observations (Data Sources 1, 2a +b, 3, 4a, 4b, 5a, 
5b, 6a, 6b, 8, and 9) was of importance as regards the seaich for patters over time that could be looked 
upon in the light of the proposition of continuous effectiveness of the motivational messages strategy. 
Important deviations from continuity of overall motivation or learning results, that could not be related 
to discontinuities in the application of the motivational strategy, would serve as indications that the 
proposition could not he, pupported. On the other hand, possible relations between changes in the 
application of the motivalional strategy and observed effects on motivational mdicators of learner 
behavior v/ere considered to support the proposition. 

Finally, while analysis is an important process in linking specific questions to particular answers 
supported by data, the case would have lost one of its most important aspects, its "wholeness", if the 
effort of carefully looking into the details described above were not carried out from the perspective of 
"embedding the empirical data at hand with a holistic understanding of the specific situation and general 
background knowledge" (Mathison, 1988, p. 17) about the phenomena being studied. 



RESULTS AND DISCUSSION 

RESEARCH RESULTS AS PROCESS 

It is beyond the scope and purpose of this paper to report in detail the results of the study and 
the argumentation that led to its major conclusions. That discussion can be found elsewhere (Visser, 
1990). What will be given here is a summary description of the results and their discussion. 

It is first of all emphasized that the results reported on in the major account of the present 
research (Visser, 1990) reflect themselves a process, rather than just the final outcome of that process, 
as would have been the case had the study been an experimental one. A picture is shown of what 
happened over time, not just in an overall sense, but revealing the detail of individual, interacting, 
developments. This is seen as a specific advantage of the case study. As it relates to a real-life context, 
the dynamics of \yhich it unveils, the knowledge it pr ovides constitutes a basis for purposeful re- 
interpretation in different contexts to which possible generalizations are intended to be made. 

If the result of the usual experimental study can be compared to the information provided by 
four single photographs, two at the beginning and two at the end, documenting different changes in an 
experimental and a control group, the result r" case study should be compared with a documentary 
film. The two processes of documenting an , .j^ iment serve different purposes. Showing the four 
pictures leaves no doubt as to the fact that the intervention reported on has had an effect (assuming 
significant difference). However, little gets known about how and why the intervention worked, and a 
blind repetition in a different context not based on knowledge about the underlying process may well 
lead to failure to reproduce the effect. The documentary film, on the other hand, is perhaps less 
convincing, in the immediate sense, as regards the overall effect of the intervention. However, it very 
carefully shows not only what has happened, but also the context in which what has happened took place. 
That more complete piece of information is likely to contribute to more successful generalizations from 
the given research. In other words, it allows the user of the research to go beyond the immediately 
given. 

The major account of the research reported on in this paper (Visser, 1990) is referenced to an 
extensive data base, the contents of which are specified and summarized in that same account. While 
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parts of that data base are, understandably, of a confidential nature, it is kept available and accessible 
for the purpose ot ftiture research. It consists of approximately 100 hours of audio recordings and two 
bookshelf lenj^ihs of written or graphic material, comprising course related documentation, circumstantial 
documentation, photographic documentation, and researcher and research assistants' field notes. 

The account given shows interesting interactions between the research activity as such and the 
instructional process in which the motivational intervention under discussion was embedded. Such 
interactions worked both ways. Research interests slightly modified the instructional process, e.g. in that 
a unit on motivation, not planned previously, was introduced in the course program, besides for its 
instructional relevance also as a means to unobtrusively collect data regarding the students' evaluation 
of motivational aspects of the course itself through exercises in which the course was taken as a concrete 
example to be analyzed. On the other hand, also the quality of ^he instructional effort gained from fact 
that simultaneously research about it was carried out. The gradually better understanding of what 
happened in applying the motivational messages strategy contributed to improvements in the use of that 
strategy. Because of the operational interconnectedness of research and instructional aims it has been 
possible to apply the various research techniques and instruments in a way that was completely natural, 
from the point of view of the participants, in the context of the instruction, as well as logical and ethical 
from the point of view of the instructor. 



THE VALIDITY ISSUE (RESEARCH QUES'nON 1) 

The extensive array of data collected basically conformed the finding of the earlier study (Visser 
& Keller, 1989) that "both on the basis of observation, and as substantiated by report and self-report, 
the motivational intervention is considered to have been successful" (p. 34), thus enhancing the validity 
of the clinical use of motivational messages in the embedded multiple-case perspective of the two cases. 

Two types of indicators were used to come to the conclusion that the strategy had been similarly 
effective in the two cases, they referred respectively to the instructional and the motivational outcome 
of the students' participation in the course. 

Comparing cases of students in the second course whose conditions of participation were similar 
to the ones that prevailed in the previous one, it was shown that the same pattern of learning task 
completion and achievement occurred, taking into account differences in entry behavior. Comparing the 
totality of the students who participated in the second course with those of the first one, it is still found 
that only slight differences occurred in student achievement, results of the second course being 
approximately five percent lower than those of the first course, whereas their distribution pattern was 
strikingly similar. Much larger differences occurred in the time needed by the students to complete their 
learning tasks. Both types of differences, however, could be accounted for, the former ones taking into 
consideration differences in entry conditions of the students, the latter ones considering the differences 
in circumstances surrounding their participation. As far as the motivational messages have played a role 
in both courses, it was therefore concluded (Visser, 1990) that they have been similarly effective in terms 
of instructional outcome of those students who completed the assignments they had undertaken to 
complete. No comparison turned o t to be possible for those students of the second course who, for 
various reasons, did not live up to their, and the researcher's, initial expectations as regards completion 
of the course assignments. Other evidence, however, indicated that even in these cases the motivational 
messages had positively influenced those students' behavior. 

An important part of the evidence in support of the validity of the motivational strategy under 
scrutiny was related, besides the inritruclional indicators, to motivational indicators. In that connection 
particular attention was paid to students' responses to the various questionnaires, to certain exercises 
that served the dual purpose of instruction and data collection, to their reactions in the open-ended end- 
of-course evaluative discussion, as well as observed verbal and attitudinai behavior of students towards 
the messages, and their reactions in interviews. Observed influences of the use of the motivational 
messages on the instructor also contributed to that evidence. 

Statistical data found, in conjunction with students' verbal references and observational data of 
a qualitative nature, interpreted in the context of the particular history of each of the two courses, 
substantiates the research proposition related to the first research question. Comparatively speaking, 
it was found that motivational indicators pointed to a stronger influence of the motivational messages 
in the second course than in the first one, despite the fact that the instructional result was less 
impressive. This is in accordance with the objectively higher degree of development, substantiated by 
data, of the use of the motivational messages in the second course. Also other aspects of the 
instructional strategy, and accompanying use of media, in the two courses show interesting differences. 
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The pattern of these differences is such that it can be explained entirely on the basis of one single 
assumption^ i.e. that the effect of motivational messages should not be looked upon in isolation, but in 
the context of a motivational system. Within that context the motivational messages constitute the main 
instrument through which the motivational facilitator manages the motivational system. A higher degree 
of effectiveness of the management of the motivational system, as was the case in the second course, 
results in changes in the patterns of differential appreciation of the importance of media and 
iastructional strategy components that can indeed be explained on the basis of the interactions within 
the motivational system. Influences of the use of the motivational messages, not only on the students, 
but also on the instructor, as observed by the research assistants, were likewise in accordance with his 
role in managing che motivational system. The scope of this paper does not allow to present the 
underlying analysis of available data in support of the above claim, as it constitutes too lengthy an 
argument. The reader is referred in that connection to the more extensive account of the study (Visser, 
1990). 

Considering the above it is concluded in the stu / that in response to the question: Did the 
motivational messages work in this instance, and did they work in a way similar to what had been the case 
in the previous course?, the answer, in the light of the above evidence, should be: Yes, they worked in a 
way that can be explained within the context of a general interpretative framework that encompasses the two 
cases. That interpretative framework served as a basis for the rationale presented in the more complete 
account of the study (Visser, 1990). One of its most particular features is its reference to the concept 
of motivational s}'^tems. The way the motivational messages were looked at in the previous study (Visser 
A Keller, 1989) should, in the light of the new, more extensive, and particularly more varied, evidence 
now available be classified as having been too narrow, too exclusively concentrating on the medium as 
an isolated thing, rather than on its functionality within a context. In other words, looking at the use 
of the motivational messages in an embedded multiple-case perspective helped raise the 
conceptualization of the phenomenon in question to a higher level. 

THE ENVIRONMENTAL ISSUE (RESEARCH QUESTION 2) 

The above considerations necessarily had to lead to a redefinition of the clinical use of 
motivational messages in terms of a strategy using a medium of u highly interactive nature within the 
context of a larger environment. That means that the functionality of that medium should also be 
analyzed from an ecological point of view. The second research question, and its related propositions, 
attended particularly to that interest. 

The environmental question poses itself at two different levels, that of the interaction between 
the medium and the persons acting within the instructional environment, and that of the interaction of 
the medium with the environment as such, i.e. the instructional habitat, jointly created and managed by 
those same persons, by the motivational messages, and by other components that are part of it. At the 
former of these two levels, there are two different types of actors. The first type is made up of the 
learners, incorporated in the instructional environment, the second kind refers to the person whose role 
it was to take the initiatives that would most likely result in the students' learning, i.e. the facilitator. 
That splits the area of interest indicated by the second research question in three sections: one dealing 
with ^at happened at the interface between messages and learners, the other one asking the same 
question as regards the interface between messages and instructor, and the final one dealing with the 
message-environment interface. The totality of these three concerns reflects the content of the second 
research question: How and why does the clinical use of motivational messages work? In connection with 
that question, and related to each of the three areas of concern, three sets of propositions were 
formulated to guide the research team in the process of collecting, analyzing, and interpreting the data. 

A large section of the major account of the study (Visser, 1990) presents evidence, with 
reference to the data base, in support of all but one of the ten research propositions related to Research 
Question 2. This evidence is fundamentally in the form of matching between, on the one hand, a 
substantiated description of how the content of the particular proposition was reflected in concrete 
features of the use of motivational messages, and, on the other hand, citations of students' spontaneous 
or solicited remarks, and descriptions of the development of their reactions to the motivational messages, 
with reference to observed and recorded behavior, using the data sources indicated in Table 2. 

It should be noted in this respect that students had a widely varying performance history while 
taking part in the course. This resulted in different ways of interaction with the medium of motivational 
messages. The "wholeness" of the information collected in the data base made it possible to follow these 
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different reactions independently from each other and to match more complex units of student behavior, 
rather than single indicators, with the particular features of the motivational messages strategy. 

A lengthy piece of evidence cited in the comprehensive account of the study (Visser, 1990) shows 
that indeed, in accordance with the first set of propositions related to the second research question, the 
motivational messages work because they help learners assess their own progress, reminding them of 
goals and intermediate goals, allowing them to correct their study behavior, enhancing their perception 
of self-control, prompting them to optimize their ime-on-task, and enhancing their satisfaction about 
accomplishments. Besides, it was found that students attached particular importance to this way of 
communicating with them. Some hard truths presented in the messages were more easily digested when 
presented this way, as opposed to if they had been communicated directly, the latter type of 
communication about such matters being perceived as 'aggressive'. 

As regards the messages-teacher interface, evidence presented supported the proposition that 
the motivational messages work because, through the process of their design and delivery, the teacl^er 
becomes more aware of the students' actual and potential problems, so that he can adjust the instruction 
accordingly. No evidence, however, was found in support of the proposition that the messages work 
because, by virtue of their prompting tne teacher into adopting a continuous mode of being motivational, 
they influence his/her in»class behavior. Evidence available, actually seemed to indicate that this was not 
the case. 

It was due to the role played by the two research assistants that valid and reliable evidence 
related to the messages-teacher interface could be collected. The comprehensive account of the study 
(Visser, 1990) describes their role in this regard in detail. It is also shown how the students' perception 
of the teacher's role in the presence of the use of motivational messages flnds a place within the earlier 
developed overall interpretative framework of a motivational system of which messages and teacher, and 
indeed numerous other components, are integrating parts. 

The same holds true for the evidence related to the last set of two propositions that the 
motivational messages work because they enhance directly (first of the two propositions) and indirectly 
(second proposition) the instructional environment. 

Using the comprehensive evidence available in the data base made it clear that the three sets 
of propositions should not be looked upon as the basis for three different rival explanations. The 
evidence available reveals the existence of a complex set of direct and indirect interactions. Fach set 
of propositions looks at a subset of these interactions, providing an incomplete, and indeed distorted, 
picture. A case in point is the view expressed in the previous study (Visser & Keller, 1989), the implicit 
research propositions of which concentrated mainly on linear interaction between the motivational 
messages and the learners. The present study reveals that that view must be paired with others, related 
to different interactions, as well as to indirect mechanisms by which the effects of the various media, 
including the motivational messages, are transmitted. Those various views are not mutually exclusive, 
but complementary, and should be placed within the context of a higher level conceptualization, that of 
a motivational systems approach. Most striking, perhaps, in that respect is a written comment by one 
of the students of the course, found on the do-you-have-any-other-observations part of a questionnaire 
and cited in support of the above contention (Visser, 1990), which says: 

Finally, I should perlxaps say that the inte^ated use of media was found to be extremely efficient and 
motivating. It was possible to create a kind of interaction between t/te different media, in such a way 
ihat the instmction, which at first had appeared to be a gigantic challenge, took on a more facilitated 
outlook and far different from the routine patterns to which we were used. 
The higher level conceptualization referred to above provided the necessary context in which the multi- 
faceted data base could be explored and interpreted in a consistent way. 



THE OPTIMAIJTY ISSUE (RESEARCH QUESTION 3) 

The flnal question addressed in the study reported on in this paper was that of optimality, 
particularly in the sense of iterative effectiveness of the strf«tegy under scrutiny. How well does it work, 
?nd can it be made to maintain its effectiveness over time under changing circumstances by adjusting 
itself to those circumstances? The relevance of that question is related to the consideration that many 
instructional interventions last for considerable periods of time. In the case of the two workshops on 
which the study was based periods of more than three months were involved. Many school based 
instructional programs are built on the assumption that students and instructor have to !ive with each 
other for at least that period of time, and often longer. Can the medium be managed by a single 
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instructor over such periods of time, and is it sufficiently sensitive to react adequately to changes in the 
environment? 

Analysis of events over time, as documented in the data base of the study (Visser, 1990), 
showed that indeed the strategy reacted adequately to changes in the environment. Two particular 
instances of quite dramatic drops in performance occurred, one related to students not meeting 
deadlines, the other one to deterioration of punctuality and assiduity. Subsequent recovery could be 
shown to follow documented intentional attention to the underlying problems of the two events through 
the motivational messages. 

Iterative effectiveness, i.e. the capability of the strategy to strive towards optimality through 
gradual improvement in the process of its application, was likewise documented (Visser, 1990) on the 
basis of evidence available in the data base. Strong arguments in that regard could be presented on the 
basis of comparison between the first and the second course. Qualitative evidence could be corroborated 
for that purpose using the statistical analysis of numerical data presented in connection with the first 
research question. 



CONCLUSIONS 



WAS THE STUDY WORTH ITS WHILE? 

The purpose of the present paper has been to draw attention to and to discuss the various 
features of the kind of approach used to respond to the research concerns that prompted the study 
reported on. It has not been my intention to inform the reader in great detail about the object of 
research and the results obtained. Interested researchers and instructional practitioners are referred in 
that regard to the comprehensive account of the study (Visser, 1990). 

In concluding this paper, the fundamental questions that should be answered are: Did the study 
live up to its expectations? Was valid knowledge created? Does that knowledge contribute to expanding 
the existing knowledge base? Can this knowledge be used by others? What are possible limitations? 

« 

THE LINK WITH THE EXISTING KNOWLEDGE BASE 

The research reported on here is rooted in an existing knowledge base. Regarding the 
motivational intervention as such, a preliminary case study had already been carried out (Visser & 
Keller, 1989), whereas a large body of knowledge regarding the underlying analysis tool, the ARCS 
model (e.g. Keller, 1987a&b), and the field of human motivation to learn in general, was also drawn 
upon (Visser, 1990). 

There is an intimate link between the clinical use of motivational messages and the ARCS model. 
In that contexi, a study which contributes to strengthening the validity of the motivational messages 
strategy, at the same time also contributes to the further validation of the ARCS model. 

The design process of the messages, as pointed out in the comprehensive account of the study 
(Visser, 1990), takes place in the four-dimensional continuum in which the motivational variables are 
defined according to the ARCS model. Learners' motivational needs, motivational objectives to be 
achieved, strategies devised to meet those objectives, and monitoring progress towards the achievement 
of the motivational objectives, can all be expressed in terms of Attention, Relevance, Confidence, and 
Satisfaction. As shown in the study, the features of the motivational messages and of their use can be 
varied such that they can be made to respond to learners' constantly changing individual and joint 
motivational requirements as assessed over time. 

Keller's (1987a4&b) ARCS model was found to be singularly suited to serve as an analysis base 
in connection with the above. It does typically not provide a standard answer to questions about what 
should be done motivationally, nor is it a model that provides its users with a simple algorithm to find 
such an answer. Models of the latter nature basically assume 'the right action for the future' to be 
deterministically related to identifiable and measurable variables of the present. Contrary to that 
position, the ARCS model assumes its users to take on a continuous problem solving mode, i.e. a way 
of involvement in motivational intervention which assumes motivation to be a continuous concern, the 
nature of which cannot be linearly derived from initial conditions, requiring analysis to be an ongoing 
process as the basis for addressing motivational problems creatively. 
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The present study emphasizes the need to approach the problem of attending to learners* 
motivational requirements from a motivational system perspective, as opposed to the conceptualization 
inherent in the previous study (Visser & Keller, 1989) in which the instructor/motivational facilitator 
was conceived as an outside agent, sending out messages, looking at what happened as a consequence 
of these messages, and taking the feedback received into account in designing and delivering subsequent 
messages. In contrast, the additional evidence, collected in the framework of the study of the second 
case, analyzed in conjunction with the evidence related to the previous case, called for a 
conceptualization in which the facilitator was looked upon as himself being part of the process. He was 
not only sender, but also receiver, and conceived as being subject to influences of the motivational 
comniunication process. Solving motivational problems could therefore, much more than on the previous 
occasion, become a joint effort of students and facilitator. The use of the ARCS model in that different 
perspective of designing and delivering motivational messages was found to be considerably more 
effective than it had previously been, as could be concluded from both the qualitative evidence collected 
about the two cases as well as from the comparative statistical analysis of the numerical data regarding 
the students* perception of the importance of the various instructional media used and of the different 
components of the instructional strategy of the two courses (Visser, 1990). 

VAUDITY, REUABIUTY, AND GENERAUZABIUTY 

In pursuing the research interest underlying the study discussed in this paper, a naturalistic, 
rather than a rationalistic, mode of inquiry was chosen. Within that mode of inquiry, a case study 
approach, and within that approach an embedded multiple-case perspective, were chosen as most suited 
to circumstances and intentions. The rationale for these choices has been presented in detail in the 
comprehensive account of the study (Visser, 1990). Major arguments contained in it have been dealt 
with in the present paper under method. Concerns regarding reliability, validity, and generalizability 
have likewise been discussed in that context. In concluding this summary account of the study, and 
looking back at the results and their discussion, can a claim that valid, reliable, and generalizable results 
were obtained be upheld? 

In response to the above question, it is recalled that one of the possible threats to validity was 
the simultaneous involvement of the same person in the role of lead researcher, as well as in those of 
instructor and motivational facilitator. Researcher bias was a real risk resulting from this combination 
of interests. On the other hand, the advantages of this construction are equally obvious. As the teacher 
of the course that served as setting for the research effort, the researcher was in the best possible, and 
most natural, position to act as a participant observer from within the teaching-learning process of the 
course. To eliminate/reduce the risks, while retaining the advantages, a number of measures was 
taken. 

A well organized, accessible data base was created using diverse data sources and different ways 
of interacting with the research environment to obtain data. The comprehensive account of the study 
(Visser, 1990) presents in extensive detail an overview and summary description of the content of the 
data base, which is kept available and accessible for future research and inspection. It also details the 
process by means of which it was established. Such a careful and comprehensive description of process 
and content is felt to be an essential requirement in the interest of establishing validity and in order to 
create a basis for replication as well as for research efforts aiming at extending the scope of the study 
reported on. 

The inclusion of two research assistants, trained and briefed to work independently from the 
lead researcher, was another essential feature in reducing researcher bias. For this claim to be a valid 
one, it is equally essential that a comprehensive account be given, and complete and reliable records kept 
(Visser, 1990), not only of the training/briefing process, but also of the implementation of the research 
assistants* tasks, and of the way their work contributed to the data base. In the case of the present study, 
and addition to the above, audio recordings were made of research debriefings, and research assistants 
produced independently a signed report, included in the data base, documenting how they perceived their 
training and their participation in the research effort, and how, from their perspective, they analyzed part 
of the evidence collected. 

The use of multiple sources of evidence, as well as multiple ways of accessing thue sources, 
were an intentional part of the research strategy, greatly facilitating corroboration of data. The extent 
and complexity of the data base reflect the degree to which the implementation of this strategy was 
taken seriously. Incidentally, in order for such extent and complexity to remain beneficial and not to 
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become a burden, sound data base management is an important requirement for studies of the kind 
under discussion to be effective. 

Unobtrusiveness in carrying out the research, both on the part of the lead researcher and the 
research assistants, was another important feature. All procedures used could be, and in fact were, 
perceived by the students as serving the interest of the course. The way this was achieved was by taking 
maximum advantage for research purposes of the instructional procedures, which were planned 
autonomously in the framework of the development of the instructional strategy of the course. At the 
same time, what was necessary from the point of view of research was planned siich that maximum 
benefit would result from it for the instruction. This way of interlinking instructional and research 
objectives has not only resulted in total unobtrusiveness, it was also a very satisfactory way of operating 
from an ethical point of view. In a country like Mozambique I would have felt it to be unethical if the 
target population of the instructional intervention in which the research was embedded would not have 
benefitted to the maximum from the instruction offered. The degree of unobtrusiveness was so high that, 
when in a final debriefing the audience was told that two of their colleagues had performed participant 
observer roles, this news came as a total surprise, as evidenced by data available in the data base and 
discussed in the comprehensive account of the study (Visser, 1990). 

Data were collected and analyzed taking into account both qualitative and quantitative concerns. 
Strong evidence could be derived from analysis of the numerical data (Visser, 1990), that was found to 
converge with evidence collected from a qualitative perspective. As subjects had not been selected and 
assigned randomly, no techniques for statistical analysis and inference could be used that would have 
been based on that assumption. The descriptive techniques used, however, in conjunction with the 
concern for naturalistic generalization, revealed eye-catching numerical differences (Visser, 1990) which, 
in conjunction with the qualitative evidence available, constituted a strong basis in support of the claims 
made in the study. 

Particularly towards the end of the course, opportu lities available in the context of the 
instructional process, were increasingly used to feed back to the audience, in an unobtrusive way, 
important provisional conclusions that started to surface in the data base, with a view to validating them 
by corroboration with audience opinion. The end-of-course open-ended evaluative discussion was a case 
in point in that regard. On other occasions, instructional procedures were used in the context of the 
teaching of a unit on motivational design, to have students debate their own motivation as regards their 
participation m the course. Those debates showed consistency between conclusions drawn on the basis 
of the data base and views expressed by the audience. Audio recordings have been included in the data 
base to provide evidence in respect of the validity of such discussions. The process was furthermore 
monitored by the research assistants, who were specifically instructed to record any instances of possible 
threat to validity of the research procedures used. 

The high degree of rigor applied in documenting the study, as evidenced by content and 
structure of the data base, was an important asset of the study under di5»cussion, particularly as regards 
reliability and generalizability. Literal replication of the study was not part of the intentions of the mode 
of inquiry chosen to pursue the interests of this research effort. Naturalistic replication, on the other 
hand, was, though, a concern. The two cases analyzed in the embedded multiple-case perspective which 
governed part of this study, constituted in fact naturalistically similar instances of the application of the 
motivational strategy under scrutiny. The fact that these two cases were not only naturalistically similar, 
but also, in some other aspects, different from each other, provided a broader basis for naturalistic 
generalizability than each of the separate cases would have established. 

One of the reasons why a case study approach was chosen for this study was the flexibility of 
that method in allowing to reveal the (fynamics of the process under investigation. The way the study 
developed proved indeed to be in accordance with that expectation. The constant interaction between 
research team and research setting, and the awareness within the research team of the need to take 
advantage of opportunities whenever they arose, has proved to be an important characteristic of the 
approach used. Adjustment of planned research procedures, in order to obtain more valid data, was 
another important feature of the case study approach. Particularly towards the end of the course, for 
instance, when a certain questionnaire fatigue became evident, anonymous collection of data was 
substituted by instructional procedures that could unobtrusively be used to obtain the same data. Also 
the approach to substantiating or insubstantiating the research propositions changed during the course 
of action. Where initially these propositions had been looked upon as possible expressions of rival 
explanations, it became gradually clear that most of them could be looked upon as providing conversion 
with a higher level perspective, lifting the messages out of their isolation and placing them in the context 
of the motivational system as a whole. 

A case study of the nature discussed here, an extensive part of the data base of which is made 
up of qualitative evidence, has its own way of presenting convincing arguments in support of its claims. 

16 



ERLC 



17 



Unfortunately, that type of description needed to communicate its rationale so as to convince others of 
the validity of conclusions drawn cannot cx)mpete with the economy of procedures used to defend 
conclusions based on statistical inference. 

This paper did not have the purpose of presenting the full rationale of the study discussed. For 
that purpose the reader is strongly advised to get exposed to the richness of information contained ui 
the comprehensive account of *he study (Visser, 1990). The present paper should be merely seen as 
a study about a study. It has communicated details about the research itself only in as far as such 
information was thought to be relevant for the purpose of exposing and discussing the mechanics of the 
research process involved. I have discussed the different features of the particular approuch used, 
emphasizing their importance for maintaining scientific rigor. The case study approach w^s found to 
be singularly suited to respond to research concerns of the nature I have described in this paper, in 
which a high degree of attention is devoted to questions of process and pattern. For that approach to 
be not only adequate, but also effective, due attention to the concerns of scientific rigor, like in the case 
of any research method, is an important requirement. 
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